Accuracy of the Berger-Exner test for detecting third-order selection bias in randomised controlled trials: a simulation-based investigation

BACKGROUND Randomised controlled trials (RCT) are highly influential upon medical decisions. Thus RCTs must not distort the truth. One threat to internal trial validity is the correct prediction of future allocations (selection bias). The Berger-Exner test detects such bias but has not been widely utilized in practice. One reason for this non-utilisation may be a lack of information regarding its test accuracy. The objective of this study is to assess the accuracy of the Berger-Exner test on the basis of relevant simulations for RCTs with dichotomous outcomes. METHODS Simulated RCTs with various parameter settings were generated, using R software, and subjected to bias-free and selection bias scenarios. The effect size inflation due to bias was quantified. The test was applied in both scenarios and the pooled sensitivity and specificity, with 95% confidence intervals for alpha levels of 1%, 5%, and 20%, were computed. Summary ROC curves were generated and the relationships of parameters with test accuracy were explored. RESULTS An effect size inflation of 71% - 99% was established. Test sensitivity was 1.00 (95% CI: 0.99 - 1.00) for alpha level 1%, 5%, and 20%; test specificity was 0.94 (95% CI: 0.93 - 0.96); 0.82 (95% CI: 0.80 - 0.84), and 0.56 (95% CI: 0.54 - 0.58) for alpha 1%, 5%, and 20%, respectively. Test accuracy was best with the maximal procedure used with a maximum tolerated imbalance (MTI) = 2 as the randomisation method at alpha 1%. CONCLUSIONS The results of this simulation study suggest that the Berger-Exner test is generally accurate for identifying third-order selection bias.